Method of sorting baggage at an airport with optimized video-encoding

ABSTRACT

A method of sorting baggage at an airport, which method comprises acquiring a plurality of digital images (IN) of a piece of baggage, which piece of baggage carries an unambiguous identification label bearing textual information about a flight, the method further comprising video coding in which a computer unit automatically detects the presence of characteristic elements of the unambiguous identification label in the digital images, computes a score for each of the digital images on the basis of a count of the characteristic elements, ranks the images as a function of their respective scores, and displays the images on a screen ( 132 ) as a function of the ranking.

TECHNICAL FIELD

A method of sorting baggage at an airport, which method comprisesacquiring a plurality of digital images of a piece of baggage that aretaken from different viewpoints, which piece of baggage carries anunambiguous identification label bearing textual information about aflight.

PRIOR ART

Such a type of method is described in Patent Application WO 2017/137106,which discloses a method in which each piece of baggage is provided withan identification label bearing lines of textual information and a barcode, which bar code enables each piece of baggage to be associatedunambiguously with a data set recorded in a baggage database that iscompiled as the baggage is being checked in.

Thus, reading the bar code or extracting textual information by using acomputer system based on optical character recognition (OCR) analysismakes it possible to track and to direct a piece of baggage in anautomated conveyor system at the departure airport and at the arrivalairport, the relevant information from the database being transmitted tothe airports in question.

In Document WO 2017/137106, textual information is extracted in theevent that it is difficult or impossible to read the bar codeautomatically. Such a difficulty occurs, for example, when the label issoiled or damaged, and the extraction is then based on comparing therecognized textual elements of the label with the data recorded in thebaggage database.

Conversely, WO 2017/137106 does not address the problems related to whenit is impossible to perform satisfactory OCR analysis and/or to accessthe data in the baggage database.

Document DE 20 2015 006 102 describes a method of sorting baggage at anairport, comprising acquiring a plurality of images of a piece ofbaggage, and video coding for displaying the images on a screen, butthat method can be improved.

SUMMARY OF THE INVENTION

To these ends, the invention provides a method of sorting baggage at anairport, which method comprises acquiring a plurality of digital imagesof a piece of baggage that are taken from different viewpoints, whichpiece of baggage carries an unambiguous identification label bearingtextual information about a flight, the method further comprising videocoding in which a computer unit automatically detects the presence ofcharacteristic elements of the unambiguous identification label in thedigital images of the piece of baggage, computes a score for each of thedigital images on the basis of a count of the characteristic elementsdetected in each image, ranks the digital images as a function of theirrespective scores, and displays said images on a screen as a function ofsaid ranking.

Displaying the digital images as a function of their ranking results inaccelerating inputting of the data by the operative at the video codingstation.

In this way, automatic selection and presentation of relevant images ofpieces of baggage simplifies the work of the operative in makingdecisions about identifying the departure airport and the destinationairport of the piece of baggage to be sorted, thereby considerablyincreasing the speed of sorting of the piece of baggage when it isimpossible to perform fully automated sorting.

The method of the invention may advantageously have the followingfeatures:

-   -   the computer unit may make a selection of the images on the        basis of their respective scores and may display the images of        the selection first on the screen to the operative;    -   the computer unit may display the images of the selection        simultaneously on the screen;    -   the computer unit may display the images of the selection        sequentially on the screen on the basis of their respective        scores;    -   the computer unit may display on the screen an enlargement of a        region of interest including a characteristic element detected        in one of the digital images;    -   the characteristic elements may be chosen from among a bar code,        an International Air Transport Association (IATA) airport code,        a flight number, and a date of a flight;    -   information that is retrieved from a database may be used to        compute the scores;    -   the computer unit may also display on the screen information        retrieved from a flights database, from among which information        an operative chooses the information that said operative enters        into the computer unit; and    -   the computer unit may also display on the screen information        retrieved from a baggage database, from among which information        an operative chooses the information that said operative enters        into the computer unit.

Thus, selecting and displaying the digital images of the piece ofbaggage can be optimized on the basis of criteria relevant for sortingthe piece of baggage, in order to facilitate and accelerateidentification of said piece of baggage by a human operative.

In addition, the decision by the human operative may be based on thecontents of a flights database, access to which and the contents ofwhich can be robust because such a flights database can be local andprepared in advance, and can be independent from a baggage database thatis remote and that is compiled as the baggage is being checked in.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention can be better understood and other advantagesappear on reading the following detailed description of animplementation given by way of non-limiting example and with referenceto the accompanying drawings, in which:

FIG. 1A shows a piece of baggage being conveyed under a gate-forminggantry equipped with cameras at an airport;

FIGS. 1B and 1C show a video coding method of the invention;

FIG. 2A shows an identification label attached to the piece of baggageof FIG. 1A, and FIGS. 2B to 2D show images of pieces of baggage used inthe method of FIGS. 1B and 1C;

FIG. 3A shows airports and associated databases; and

FIGS. 3B and 3C are flow charts showing a sorting method of theinvention.

DESCRIPTION OF AN IMPLEMENTATION

A method of the invention for sorting baggage at an airport is shown byFIGS. 1A to 3C.

During a journey by air from an airport A to an airport B, or from anairport C to airport A, such journeys being respectively referencedV_(AB) and V_(CA) as shown in FIG. 3A, a piece of baggage conveyed atairport A is identified by means of an unambiguous identification label100 attached to said piece of baggage.

The label bears a bar code and is printed out at the time the piece ofbaggage is checked in, in such a manner as to associate the bar codewith a set of information relating to the piece of baggage, such as aflight number, an airline, or a check-in number, which set ofinformation is recorded in a baggage database (300) compiled as thebaggage is being checked in at a departure airport and is thentransmitted between the airports in question by means of a computernetwork.

In addition to the bar code 220, said label 100 bears textualinformation that can be useful for tracking the piece of baggage and fordirecting it towards its destination, such as an IATA airport codecomposed of a three-letter acronym 222 corresponding to an airport, anda flight number 226 composed of an encoded name 127 of an airline and ofa flight number 228 internal to that airline, a date 224, and a city230, as shown in FIG. 2A.

In the present implementation, the piece of baggage 110 is transportedwithin an airport by a conveyor 102 of a conveyor system 101 controlledby an automatic sorting unit 140 and goes through or past anidentification system 130 comprising a reader device 125 for reading thebar code 220 on the label 100 with a view to sorting said piece ofbaggage as a function of the data associated with the bar code in thebaggage database 300.

The identification system 130 also includes a gate-forming gantry 131equipped with a plurality of cameras 120 arranged to acquire digitalimages IN of the piece of baggage 110 as seen from different angles, inorder to maximize the chances of having usable images of the label 100.

In the event the automatic reading (reading of the bar code 220 or OCRreading of the textual information) fails to sort the piece of baggage,it is possible to use the digital images IN to perform sorting by videocoding, in which the digital images are presented to a human operativewho identifies information that corresponds to the textual informationon the label and inputs it manually into a computer system in order todirect the piece of baggage in the conveyor system 101.

For example, a gate-forming gantry may have ten cameras, each of whichacquires a series S(IN) of, e.g. ten, digital images IN of the piece ofbaggage, and can thus produce a set E(IN) of one hundred digital imagesIN that can be presented to the human operative Op.

Such a number of images guarantees with a high level of certainty thepresence of at least one digital image enabling the operative to make adecision about the information to be entered manually, but, in theabsence of optimization of the presentation of the images, it can benecessary for the operative to view a large number of images beforefinding one in which the label of the piece of baggage is present andlegible.

The label might be hidden from some of the cameras by the piece ofbaggage itself, e.g. if it is under said baggage, and, even if it isvisible, its orientation might make it illegible in the majority of thedigital images.

In addition, a label might be only partially visible in a digital imageIN, or indeed it might be damaged, as in FIGS. 2B and 2C.

Thus, each digital image IN considered individually has a highlikelihood of being unusable for sorting by video coding.

In accordance with the invention, a computer unit 135 puts the digitalimages into an order of priority so as to help the human operative incharge of the video coding by giving priority to presenting the imagesthat are most likely to contain information that is useful for the videocoding.

In a first step S01 of the video coding VCS shown in FIGS. 1B, 1C, and3B, the computer unit 135 collects the set E(IN) of digital imagesacquired by the cameras 120 during a step S01, and then automatically,for each of the digital images, performs search software analysis so asto search for characteristic elements of the identification label 100during a step S02, in such a manner as to detect in the digital imagethe presence, or absence, of such characteristic elements.

Absence of any one of these characteristic elements in an image can bededuced from negative detection of the element, i.e. absence ofdetection of that element in spite of the search software analysis.

The characteristic elements that are searched for are chosen to beindicative of the probability that an image IN shows the label 100, andin particular shows the textual information on which the human operativebases the identification; said characteristic elements may be a tripletof characters 222 potentially corresponding to an IATA airport code, aline of characters 226 potentially corresponding to a flight number, aline of characters 226 potentially corresponding to a flight number inthe vicinity of a triplet 222 of characters, a line of characters 224potentially corresponding to the date of the flight, any line ofcharacters or any graphics usually present on a label affixed to baggageat an airport for identification purposes, a quality of automaticreading of said elements, a bar code 220, or a blank region 223 in thevicinity of a triplet 222 of characters potentially corresponding to anIATA airport code.

The potential correspondences mentioned above are to be taken in thefollowing sense: the search software analysis can identify textualcharacters and lines of such characters in terms of numbers, in terms oftypes (such as digits, letters, or other types) and/or in terms ofsequences that correspond to textual information that is usually to befound on the label, but it does not necessarily make said elementscorrespond to information such as an existing flight number, with anysuch interpretation of the identified elements being entrusted to thehuman operative in the context of the video coding.

For example, a line comprising two letters followed by three digits maybe considered to correspond potentially to a flight number such as theflight number 226 shown in FIG. 2A.

On the basis of a count of the presences and of the absences of thecharacteristic elements, the computer unit then computes a score foreach of the digital images during a step S03, which score may, forexample, be obtained by assigning a positive index to eachcharacteristic element detected, and thus considered to be present inthe image, a negative index to each characteristic element not detected,and thus considered to be absent from the image, and then by summing theindices for each digital image, so that the digital images that have thehighest scores are the most likely to show the textual information ofthe label and thus to be useful to the operative.

Computing the score may be based on weighting rules that are determinedempirically or may be performed on the basis of automatic learning, e.g.by using a neural network.

The score may also be computed directly by neural analysis of the pixelimage.

The score may also be computed using information retrieved from adatabase, such as the baggage database when said baggage database isavailable or a flights database, e.g. by increasing the index assignedto a potential correspondence that actually corresponds to an item ofdata present in said database.

Once the scores have been established, the computer unit ranks thedigital images IN as a function of their respective scores during a stepS04 and displays said scores on a screen 132 as a function of thatranking during a step S05.

In this document, the function of displaying on a screen is not limitedto displaying by a single display unit, but rather it includesdisplaying on a plurality of physically distinct display units, such as,for example, separate computer monitors.

In addition, displaying the digital images is not limited to displayingthe images in full, but rather it includes, for example, displaying someof the images and/or images that have undergone one or more digitalprocessing operations such as an enlargement or a change in thecontrast.

In the above-described method of computing the scores, in which methodthe higher the score of a digital image the higher the probability thatthat image shows elements useful to the operative, the computer unit maymake a selection of images by keeping only those images that have thehighest scores and by displaying on the screen only the digital imagesof the selection, or by giving priority to displaying them.

The computer unit may display all of the digital images of the selectionsimultaneously on the screen, e.g. four images as shown by Sel-1 in FIG.1C.

Alternatively, the computer unit may display on the screen the digitalimages of the selection in sequential manner in decreasing order of thescores of the digital images, as shown by Sel-2 in FIG. 1C, in such amanner as to present the images that are the most likely to be useful tothe operative first, it being possible for the operative to scrollthrough the images on the screen manually.

In addition to or instead of a given image IN, the computer unit mayalso display on the screen an enlargement of a region of interest ZIdefined as a region of the image in which at least one of thecharacteristic elements indicative of the probability of the imageshowing the label is present.

To this end, the computer unit may, for example, define a region ofinterest as a region of the image having a certain size and centered onat least one of the characteristic elements of the label 100, as shownin FIG. 2D in which a region of interest ZI is centered on the bar code220 that is visible in the digital image IN.

On viewing the displayed images, selected because they have highprobabilities of containing information that is useful to the humanoperative Op of FIG. 1B, said human operative identifies the textualinformation borne by the label and enters data representative of saidinformation manually into a computer system via an input terminal 134that may, for example be a keyboard, a touch tablet, or a voice system,during a step S06 that ends the video coding.

To assist the operative, the computer unit 135 may also display on thescreen 132 information provided by a database and likely to correspondto the piece of baggage, the images of which are displayed on thescreen, and preferably information also corresponding to the firstcharacters entered manually by the operative.

A first database likely to contain information useful to sorting thepiece of baggage is the baggage database 300.

Alternatively, a flights database 310 may provide such information.

Such a database 310 is local to the airport where the baggage is to befound, and lists the flights to and from that airport during a givenperiod of activity, e.g. one day or two or more consecutive days ofactivity at the airport, and that includes in particular, the day onwhich a piece of baggage is sorted in accordance with the invention,said flights database associating, for example, for each of the flights,an IATA airport code with a flight number composed of an airline codeand of a flight number internal to the airline.

Since said flights database 310 is local, it offers better stability andavailability, and can be known much further in advance than a baggagedatabase 300, and using it therefore makes it possible to achieve muchgreater operational reliability than using the baggage database.

In the method shown in FIG. 3C, the priority baggage identificationmethod is automatically reading S10 the bar codes on the labels 100attached to the baggage 110 for the purpose of unambiguously associatingthe bar code of each piece of baggage with data from the baggagedatabase 300, from which representative items of data are transmitted tothe automatic sorting unit 140 during a step S12 so that the automaticsorting unit 140 controls the conveyor system 101 in such a manner as todirect the piece of baggage 110 appropriately during a step S14.

In the event of failure N10 to identify a piece of baggage by means ofthe bar code of the corresponding label, e.g. in the event it isimpossible to connect to the baggage database 300 or in the event thelabel is damaged, the digital images IN can be used in the context of anautomated OCR analysis S20.

In this OCR analysis S20, the computer unit 135 may compare the chainsof characters recognized on the label 100 with the information from theflights database 310 and, in the event of a match, transmit the datarepresentative of the information on the label to the automatic sortingunit 140 during a step S22, in such a manner that said automatic sortingunit controls the conveyor system 101 in such a manner as to direct thepiece of baggage appropriately during a step S24.

In the event of no match being found N20 or of a confidence index thatis too low in the OCR analysis S20, the video coding VCS defined by theabove-described steps S01 to S06 is implemented, during which videocoding the computer unit 135 displays on the screen 132 information fromthe flights database 310 or from the baggage database 300 for assistingthe human operative Op.

In response to the human operative inputting data necessary for sortingthe piece of baggage, steps S22 and S24 are executed as in the OCRanalysis S20, without all of the selected images having necessarily beendisplayed.

Thus, the method for sorting baggage at an airport with optimized videocoding makes it possible to solve the problems related to readinginformation present on the label and the problems related to using abaggage database.

Naturally, the present invention is in no way limited to theabove-described implementation, which can undergo modifications withoutgoing beyond the ambit of the invention.

1. A method of sorting baggage at an airport, which method comprises:acquiring a plurality of digital images of a piece of baggage that aretaken from different viewpoints, which piece of baggage carries anunambiguous identification label bearing textual information about aflight, and video coding, wherein, in the video coding, a computer unitautomatically detects the presence of characteristic elements of theunambiguous identification label in the digital images of the piece ofbaggage, computes a score for each of the digital images on the basis ofa count of the characteristic elements detected in each image, ranks thedigital images as a function of their respective scores, and displayssaid images on a screen as a function of said ranking.
 2. The sortingmethod according to claim 1, wherein the computer unit makes a selectionof the images on the basis of their respective scores and displays theimages of the selection first on the screen to the operative.
 3. Thesorting method according to claim 2, wherein the computer unit displaysthe images of the selection simultaneously on the screen.
 4. The sortingmethod according to claim 2, wherein the computer unit displays theimages of the selection sequentially on the screen on the basis of theirrespective scores.
 5. The sorting method according to claim 4, whereinthe computer unit displays on the screen an enlargement of a region ofinterest including a characteristic element detected in one of thedigital images.
 6. The sorting method according to claim 5, wherein thecharacteristic elements are chosen from among a bar code, an IATAairport code, a flight number, and a date of a flight.
 7. The sortingmethod according to claim 6, wherein information that is retrieved froma database is used to compute the scores.
 8. The sorting methodaccording to claim 7, wherein the computer unit also displays on thescreen information retrieved from a flights database, from among whichinformation an operative chooses the information that said operativeenters into the computer unit.
 9. The sorting method according to claim8, wherein the computer unit also displays on the screen informationretrieved from a baggage database, from among which information anoperative chooses the information that said operative enters into thecomputer unit.
 10. The sorting method according to claim 1, wherein thecomputer unit displays on the screen an enlargement of a region ofinterest including a characteristic element detected in one of thedigital images.
 11. The sorting method according to claim 1, wherein thecharacteristic elements are chosen from among a bar code, an IATAairport code, a flight number, and a date of a flight.
 12. The sortingmethod according to claim 1, wherein information that is retrieved froma database is used to compute the scores.
 13. The sorting methodaccording to claim 1, wherein the computer unit also displays on thescreen information retrieved from a flights database, from among whichinformation an operative chooses the information that said operativeenters into the computer unit.
 14. The sorting method according to claim1, wherein the computer unit also displays on the screen informationretrieved from a baggage database, from among which information anoperative chooses the information that said operative enters into thecomputer unit.
 15. The sorting method according to claim 2, wherein thecomputer unit displays on the screen an enlargement of a region ofinterest including a characteristic element detected in one of thedigital images.
 16. The sorting method according to claim 2, wherein thecharacteristic elements are chosen from among a bar code, an IATAairport code, a flight number, and a date of a flight.